Method and system for classifying brain signals in a BCI using a subject-specific model

ABSTRACT

A method or system for classifying brain signals in a BCI. The system comprises a model building unit for building a subject-independent model using labelled brain signals from a pool of subjects.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a National Phase Patent Application of and claimspriority to International Application Number PCT/SG2008/000192, filed onMay 26, 2008.

FIELD OF INVENTION

The present invention relates broadly to a method and system forclassifying brain signals in a BCI system, and to a data storage mediumhaving stored thereon computer code means for instructing a computersystem to execute a method for classifying brain signals in a BCI.

BACKGROUND

Brain computer interface (BCI) [J. R. Wolpaw, N. Birbaumer, D. J.McFarland, G. Pfurtscheller, and T. M. Vaughan, Brain-computerinterfaces for communication and control, Clinical Neurophysiology, vol.113, pp. 767-791, 2002.; E. A. Curran and M. J. Strokes, Learning tocontrol brain activity: A review of the production and control of EEGcomponents for driving brain-computer interface (BCI) systems, Brain andCognition, vol. 51, pp. 326-336, 2003.] functions as a directcommunication pathway between a human brain and an external device. Asit directly uses the electrical signatures of the brain's activity forresponding to external stimuli, it is particularly useful for paralyzedpeople who suffer from severe neuromuscular disorders and are henceunable to communicate through the normal neuromuscular pathway. Theelectroencephalogram (EEG) is one of the widely used techniques out ofmany existing brain signal measuring techniques due to its advantagessuch as its non-invasive nature and its low cost.

Farwell and Donchin [L. A. Farwell and E. Donchin, Talking off the topof your head: toward a mental prosthesis utilizing event-related brainpotential, Electroencephalography and Clinical Neurophysiology, vol. 70,pp. 510-523, 1988.] first demonstrated the use of P300 for BCIs in aso-called oddball paradigm. P300 is an endogenous, positive polaritycomponent of the evoke-related-potential (ERP) elicited in the brain inresponse to infrequent/oddball auditory, visual or somatosensorystimuli. In the oddball paradigm, the computer displays a matrix ofcells representing different letters, and flashes each row and columnalternately in a random order. FIG. 1 shows an example of the matrix ofcells 100 displayed by a computer in the oddball paradigm. A user tryingto input a letter is required to pay attention to the target letter fora short while. In this process, when the row or column containing theintended letter flashes, a P300 will be elicited in the subject's EEG,which is then identified by using signal processing and machine learningalgorithms.

One problem with using the P300 in BCIs is that large inter-subjectvariations exist among P300 of different subjects. For example, the P300amplitude and latency vary among both normal and clinical populations.Such variations have been linked with individual differences incognitive capability. Therefore, from the pattern recognition viewpoint,computational P300 classification models built for one subject does notaccurately apply to another subject. To solve this problem, mostP300-based BCIs usually first perform a special training session tolearn a subject-specific classification model. In that special trainingsession, a subject is required to follow instructions and focus on aparticular cell visually at a given time while his or her EEG is beingrecorded. Subsequently, certain computer algorithms are implemented toperform the signal analysis and to learn a subject-specificclassification model based on the recorded EEG. One problem with thespecial training session described above is that it is normallycomplicated and tedious, making most P300-based BCIs user-unfriendly.Furthermore, the requirement for the special training sessions makes thepractical implementation of P300-based BCIs difficult.

Hence, in view of the above, there exists a need for a method and systemfor classifying brain signals in a BCI which seek to address at leastone of the above problems

SUMMARY

In accordance with a first aspect of the present invention there isprovided a method for classifying brain signals in a BCI, the methodcomprising the step of building a subject-independent model usinglabelled brain signals from a pool of subjects.

The method may further comprise the step of building an initialsubject-specific model based on a set of feature vectors extracted fromunlabelled brain signals from a new subject, applying both thesubject-independent model and the initial subject-specific model forclassifying the unlabelled brain signals.

The method may further comprise the step of adapting the initialsubject-specific model using one or more of a group consisting ofsubsequent segments of unlabelled brain signals from the new subject,the subject-independent model and the initial subject-specific model.

The adapting may be performed until the subject-specific model achievesa consistent confidence score and subsequently the adapted subjectspecific model is used to give the classification of the brain signals.

The step of building the subject-independent model using labelled brainsignals from a pool of subjects may further comprise the steps ofacquiring the labelled brain signals from the pool of subjects;preprocessing the acquired labelled brain signals; constructing a set offeature vectors with their corresponding labels from the preprocessedbrain signals; and building the subject-independent model by finding aweight vector for a linear combination of each feature vector tomaximize the posterior probability that a P300 is evoked or not evokedgiven a feature vector.

The step of building the initial subject-specific model may comprise thesteps of acquiring the unlabelled brain signals from the new subject;segmenting the acquired unlabelled brain signals; preprocessing theacquired unlabelled brain signals; extracting a set of feature vectorsfrom the preprocessed unlabelled brain signals; and classifying thefirst segment of the unlabelled brain signals using thesubject-independent model to build the initial subject-specific model.

The step of acquiring the labelled brain signals from the pool ofsubjects may further comprise the steps of providing a pre-defined setof stimuli in rows and columns; repeatedly activating the stimuli inrounds, wherein in each round, each row or column of stimuli isactivated once; acquiring brain signals from the pool of subjects witheach subject focused on a known stimulus; and labelling the acquiredbrain signals from the pool of subjects using the label of the knownstimulus to give the labelled brain signals.

The step of acquiring the unlabelled brain signals from the new subjectmay further comprise the steps of providing a pre-defined set of stimuliin rows and columns; repeatedly activating the stimuli in rounds,wherein in each round, each row or column of stimuli is activated once;and acquiring the unlabelled brain signals from the new subject with thesubject focused on an unknown stimulus.

The step of preprocessing the acquired labelled brain signals mayfurther comprise the steps of implementing a low-pass filtering of theacquired labelled brain signals using an optimal cutoff frequency;down-sampling the filtered brain signals by averaging every fiveconsecutive samples to a single sample; and removing ocular artifactsfrom the downsampled brain signals.

The step of segmenting the acquired unlabelled brain signals may furthercomprise the step of including brain signals collected for more than onestimulus in the first segment and including brain signals collected forone stimulus in each of the subsequent segments.

The step of adapting the initial subject-specific model using one ormore of a group consisting of subsequent segments of unlabelled brainsignals from the new subject, the subject-independent model and theinitial subject-specific model may further comprise the steps ofiteratively a) classifying the feature vectors corresponding to thesubsequent segment of the unlabelled brain signals using thesubject-independent model; b) classifying the feature vectorscorresponding to the subsequent segment of the unlabelled brain signalsusing the initial subject-specific model; c) evaluating a confidencescore for the subject-independent model; d) evaluating a confidencescore for the initial subject-specific model; e) classifying the featurevector corresponding to the subsequent segment of the unlabelled brainsignals using the model with a higher confidence score; f) determiningif the initial subject-specific model has achieved a consistentconfidence score; g) adapting the initial subject-specific model usingclassification results from the model with a higher confidence score ifthe subject-specific model has not achieved a consistent confidencescore; and repeating steps a) to g) with the adapted initialsubject-specific model as the initial subject-specific model.

The step of evaluating the confidence score for the subject-independentmodel may further comprise the steps of evaluating a posteriorprobability that a P300 is evoked given the feature vector for each rowof stimuli; evaluating a posterior probability that a P300 is evokedgiven the feature vector for each column of stimuli; determining thedifference between the highest posterior probability among the rows ofstimuli and the next highest posterior probability among the rows ofstimuli to give a saliency of the highest posterior probability andmultiplying said saliency to said difference; determining the differencebetween the highest posterior probability among the columns of stimuliand the next highest posterior probability among the columns of stimulito give a saliency of the highest posterior probability and multiplyingsaid saliency to said difference; combining the product of the saliencyand the difference for the rows of stimuli and the columns of stimuli toevaluate a confidence score for the subject-independent model.

The step of evaluating the confidence score for the initialsubject-specific model may further comprise the steps of evaluating aposterior probability that a P300 is evoked given the feature vector foreach row of stimuli; evaluating a posterior probability that a P300 isevoked given the feature vector for each column of stimuli; determiningthe difference between the highest posterior probability among the rowsof stimuli and the next highest posterior probability among the rows ofstimuli to give a saliency of the highest posterior probability andmultiplying said saliency to said difference; determining the differencebetween the highest posterior probability among the columns of stimuliand the next highest posterior probability among the columns of stimulito give a saliency of the highest posterior probability and multiplyingsaid saliency to said difference; combining the product of the saliencyand the difference for the rows of stimuli and the columns of stimuli toevaluate a confidence score for the initial subject-specific model.

The step of determining if the initial subject-specific model hasachieved a consistent confidence score may further comprise the steps ofdetermining if the confidence score of the initial subject-specificmodel for a current segment of brain signals from the new subject isgreater than a first threshold; determining if the standard deviation ofthe confidence scores of the initial subject-specific model for the lastk segments of brain signals from the new subject is less than a secondthreshold; and determining that the initial subject-specific model hasachieved a consistent confidence score if the confidence score of theinitial subject-specific model for a current segment of brain signals isgreater than said first threshold and the standard deviation of theconfidence scores of the initial subject-specific model for the last ksegments of brain signals is less than said second threshold.

The method may further comprise the step of identifying the unknownstimulus by identifying the row and the column in which the unknownstimulus lies wherein the stimulus in said row and said column resultsin a maximum averaged posterior probability that P300 is evoked given afeature vector.

In accordance with a second aspect of the present invention there isprovided a system for classifying brain signals in a BCI, the systemcomprising a model building unit for building a subject-independentmodel using labelled brain signals from a pool of subjects.

The system may further comprise a second model building unit forbuilding an initial subject-specific model based on a set of featurevectors extracted from unlabelled brain signals from a new subject,applying both the subject-independent model and the initialsubject-specific model for classifying the unlabelled brain signals.

The system may further comprise a model adapting unit for adapting theinitial subject-specific model using one or more of a group consistingof subsequent segments of unlabelled brain signals from the new subject,the subject-independent model and the initial subject-specific model.

The adapting may be performed until the subject-specific model achievesa consistent confidence score and subsequently the adapted subjectspecific model is used to give the classification of the brain signals.

The system may further, comprise a stimulation unit comprising a set ofstimuli in rows and columns; wherein the stimulation unit repeatedlyactivates the stimuli in rounds, such that in each round, each row orcolumn of stimuli is activated once; an acquisition unit for acquiringbrain signals; and a preprocessing unit for preprocessing the acquiredbrain signals.

In accordance with a third aspect of the present invention there isprovided a data storage medium having stored thereon computer code meansfor instructing a computer system to execute a method for classifyingbrain signals in a BCI, the method comprising the step of building asubject-independent model using labelled brain signals from a pool ofsubjects.

The method may further comprise the step of building an initialsubject-specific model based on a set of feature vectors extracted fromunlabelled brain signals from a new subject, applying both thesubject-independent model and the initial subject-specific model forclassifying the unlabelled brain signals.

The method may further comprise the step of adapting the initialsubject-specific model using one or more of a group consisting ofsubsequent segments of unlabelled brain signals from the new subject,the subject-independent model and the initial subject-specific model.

The adapting may be performed until the subject-specific model achievesa consistent confidence score and subsequently the adapted subjectspecific model is used to give the classification of the brain signals.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the invention will be better understood and readilyapparent to one of ordinary skill in the art from the following writtendescription, by way of example only, and in conjunction with thedrawings, in which:

FIG. 1 shows an example of a matrix of cells displayed by a computer inan oddball paradigm.

FIG. 2 shows a flowchart illustrating a zero-trained P300 identificationtechnique according to an embodiment of the present invention.

FIG. 3 shows a flowchart illustrating an unsupervised modeling andclassification technique according to an embodiment of the presentinvention.

FIG. 4 shows a plot illustrating the performances of different P300identification techniques.

FIGS. 5A-J show graphs illustrating the accuracy of different P300identification techniques.

FIGS. 6A-J show graphs illustrating the standard deviation of the P300identification accuracy of supervised SSMs and adapted SSMs according toan embodiment of the present invention.

FIG. 7 shows a plot illustrating the classification accuracy ofdifferent P300 identification techniques.

FIG. 8 illustrates a schematic block diagram of a system for classifyingbrain signals in a BCI according to an embodiment of the presentinvention.

FIG. 9 illustrates a schematic block diagram of a computer system onwhich the method and system of the example embodiments can beimplemented.

DETAILED DESCRIPTION

Some portions of the description which follows are explicitly orimplicitly presented in terms of algorithms and functional or symbolicrepresentations of operations on data within a computer memory. Thesealgorithmic descriptions and functional or symbolic representations arethe means used by those skilled in the data processing arts to conveymost effectively the substance of their work to others skilled in theart. An algorithm is here, and generally, conceived to be aself-consistent sequence of steps leading to a desired result. The stepsare those requiring physical manipulations of physical quantities, suchas electrical, magnetic or optical signals capable of being stored,transferred, combined, compared, and otherwise manipulated.

Unless specifically stated otherwise, and as apparent from thefollowing, it will be appreciated that throughout the presentspecification, discussions utilizing terms such as “calculating”,“generating”, “building”, “adapting”, “acquiring”, “preprocessing”,“constructing”, “segmenting”, “classifying”, “providing”, “activating”,“labelling”, “implementing”, “down-sampling”, “removing”, “predicting”,“evaluating”, “determining”, “combining”, “identifying” or the like,refer to the action and processes of a computer system, or similarelectronic device, that manipulates and transforms data represented asphysical quantities within the computer system into other data similarlyrepresented as physical quantities within the computer system or otherinformation storage, transmission or display devices.

The present specification also discloses apparatus for performing theoperations of the methods. Such apparatus may be specially constructedfor the required purposes, or may comprise a general purpose computer orother device selectively activated or reconfigured by a computer programstored in the computer. The algorithms and displays presented herein arenot inherently related to any particular computer or other apparatus.Various general purpose machines may be used with programs in accordancewith the teachings herein. Alternatively, the construction of morespecialized apparatus to perform the required method steps may beappropriate. The structure of a conventional general purpose computerwill appear from the description below.

In addition, the present specification also implicitly discloses acomputer program, in that it would be apparent to the person skilled inthe art that the individual steps of the method described herein may beput into effect by computer code. The computer program is not intendedto be limited to any particular programming language and implementationthereof. It will be appreciated that a variety of programming languagesand coding thereof may be used to implement the teachings of thedisclosure contained herein. Moreover, the computer program is notintended to be limited to any particular control flow. There are manyother variants of the computer program, which can use different controlflows without departing from the spirit or scope of the invention.

Furthermore, one or more of the steps of the computer program may beperformed in parallel rather than sequentially. Such a computer programmay be stored on any computer readable medium. The computer readablemedium may include storage devices such as magnetic or optical disks,memory chips, or other storage devices suitable for interfacing with ageneral purpose computer. The computer readable medium may also includea hard-wired medium such as exemplified in the Internet system, orwireless medium such as exemplified in the GSM mobile telephone system.The computer program when loaded and executed on such a general-purposecomputer effectively results in an apparatus that implements the stepsof the preferred method.

Embodiments of the present invention employ a zero-trained subject EEGmodeling and classification technique to address the above-mentionedproblems. The inventors have recognized that despite the existence oflarge inter-subject variations within the P300 of different subjects,there remain common characteristics within the P300 of differentsubjects. One example of such a common characteristic is the positivepeak in the P300 after 300 ms from the time an external stimulus isapplied. Compared with a P300 model learned from one specific subject, aP300 model learned from a pool of subjects can be more capable incapturing the common characteristics. Such a subject model learned froma pool of subjects in the example embodiments can be referred to as thesubject-independent model (SIM) because it is independent of anyspecific subject and can capture the common P300 characteristics. Such aSIM can identify the P300 of a new subject without special training andhence has a higher potential in classifying EEG of people in generalwithout the special training.

Although the SIM in the example embodiments is capable of identifyingP300 of a new subject without special training, the identificationaccuracy may be lower than that of a supervised subject specific model(SSM) learned from a subject's labelled EEG. This can be attributed tothe fact that the SIM captures the common P300 characteristics insteadof the subject-specific P300 characteristics.

The example embodiments can additionally make use of the EEG of a newsubject and accordingly capture the subject-specific P300characteristics through an unsupervised learning process. Such exampleembodiments include a new P300 modeling and identification techniquethat adapts a SIM to a SSM through an unsupervised learning process.Given labelled EEG of a pool of subjects and unlabelled EEG of a newsubject, the EEG of the new subject is initially classified by using aSIM. A SSM is then built based on the initially classified EEG segmentof the new subject and the corresponding labels predicted by the SIM.Subsequently, both the SIM and the newly built SSM are deployed toclassify the ensuing subject EEG. The classification results can then bedetermined according to the model with the higher confidence score. Inthis way, the SSM can be iteratively updated by incorporating the newlyclassified subject EEG which is dependent on the ensuing EEG of the newsubject and the corresponding labels predicted by either the SIM or theSSM, depending on their classification confidence score. This adaptationprocess can be terminated in such embodiments when the adapted SSM hasachieved consistency.

In the example embodiments, collection of the EEG from the subjectsusing a P300-based word speller system is performed. Further details ofthe P300-based word speller•system can be found in [T. Manoj, C. Guan,and W. Jiankang. Robust classification of EEG signal for brain-computerinterface, IEEE Transactions on Neural Systems and RehabilitationEngineering, vol. 14, no. 1, pp. 24-29, 2006.], details of which areincorporated herein by cross-reference.

During the collection of the EEG in the example embodiments, subjectsare equipped with an electro-cap that has 64 electrodes mounted. The EEGcollected from the subjects is first amplified by for example, aNeuroscan amplifier called SynAmps2 and then piped to a server by forexample, the Neuroscan software. The SynAmpls2 has 64 mono-polarchannels through which the measured EEG is transmitted. In one example,24 out of the 64 channels are automatically selected and the EEGsampling rate is set at 250 Hz.

Furthermore, during the EEG collection stage in the example embodiments,subjects sit in front of a 6×6 matrix of cells with each cell displayinga character as shown in FIG. 1 and the six rows and columns flashsuccessively and randomly. Subjects are required to focus on onespecific cell during a flashing round whereby each round is defined bythe flashing of each row or column once. When a particular row or columnflashes, a corresponding stimulus code is generated in a time-lockedfashion and divides the collected EEG into epochs of 500 ms startingfrom the time the stimulus appears. The focused cell corresponding toone row flash and one column flash within each round can be determinedthrough the identification of the P300 within the epoched EEG.

FIG. 2 shows a flowchart illustrating a zero-trained P300 identificationtechnique 200 according to an embodiment of the present invention. Theinput to the technique 200 in FIG. 2 is a set of labelled EEG, E,collected from a pool of subjects and unlabelled EEG, E′, collected,from a new subject. The collection process has been described above.

In step 202, the labelled EEG from a pool of subjects, E, ispreprocessed. In one example, the EEG is preprocessed by firstimplementing a low-pass filtering of the EEG using an optimal cutofffrequency [C. Guan, X. Zhu, S. Ranganatha, M. Thulasidas, and J. Wu,Robust classification of event-related potential for brain-computerinterface, Int. Conf. Adv. Medical Signal Info. Processing, pp. 321-326,2004.]. The filtered EEG is then down-sampled for example, by averagingevery five consecutive EEG samples to a single EEG sample. Suchdown-sampling reduces the data size and at the same time speeds up theensuing EEG processing significantly.

Ocular artifacts are then removed by treating the sampled EEG y(n) as alinear superposition of the measured EOG u(n) and the real EEG w(n)according to Equation (1). In Equation (1), N is the number of sites atwhich the EOG measurement is done. In one example, N is equal to two.

$\begin{matrix}{{y(n)} = {{\sum\limits_{i = 1}^{N}{b_{i}{u_{i}(n)}}} + {w_{i}(n)}}} & (1)\end{matrix}$

In one example, the EOG is removed by using a difference model whichremoves the inter-sample correlations of the required EEG w(n) as shownin Equation (2). In Equation (2), n′=n−1. Since the dynamic range of wis small in comparison to u, the propagation constants b_(i) can becomputed through the least square minimization. Further details of thedifference model can be found in T. Manoj, C. Guan, and W. Jiankang.Robust classification of EEG signal for brain-computer interface, IEEETransactions on Neural Systems and Rehabilitation Engineering, vol. 14,no. 1, pp. 24-29, 2006, details of which are incorporated herein bycross-reference.

$\begin{matrix}{{y(n)} = {{y( n^{\prime} )} + {\sum\limits_{i = 1}^{N}{b_{i}( {{u_{i}(n)} - {u_{i}( n^{\prime} )}} )}} + {w_{i}(n)} - {w_{i}( n^{\prime} )}}} & (2)\end{matrix}$

In step 204, feature extraction is performed such that the preprocessedEEG is converted into a set of feature vectors together with theircorresponding labels. In step 206, a SIM is built based on the set offeature vectors together with their corresponding labels obtained instep 204. Further details of steps 204 and 206 are as follows.

In the example embodiment, a total of 12 flashes intensify in a randomorder within each round. In each round, the flashing of a particular rowor column within which the focused cell lies results in a P300 evoked inthe subject whereas the flashing of the remaining rows or columns doesnot result in any P300 being evoked in the subject. Therefore, the P300identification can be treated as a two-class classification problem. Tofacilitate the P300 identification, the preprocessed EEG E_(c×s)collected within each trial is first converted into a feature vector asshown in Equation (3). In Equation (3), x(i) refers to the EEG collectedfrom the i-th selected channel and the parameter c refers to the numberof channels selected. In one example, the EEG collected from the i-thselected channel is composed of s EEG signals sampled between 150 ms and500 ms after each flash and the number of channels selected is 24.x=[x(1)^(T) , . . . ,x(i)^(T) , . . . ,x(c)^(T)]^(T)  (3)

In the example embodiment, it can be assumed that the EEG feature vectorx (either with or without P300) has a multivariate Gaussian distributionwith mean μ_(i) and covariance Σ_(i) according to Equation (4). InEquation (4), x refers to the feature vector converted from E_(c×s) andp=c×s is equal to the dimension of the feature vector x. The parameterθ_(b) represents the hypothesis that the EEG contains P300 when i=1 andrepresents the hypothesis that the EEG does not contain P300 when i=2.p(x|θ_(i)) refers to the probability that the feature vector x isobtained given the hypothesis θ_(i). In addition, parameters μ_(i) andΣ_(i), i=1, 2 refer to the mean and the covariance of the feature vectorx with and without the presence of P300, respectively.

$\begin{matrix}{{{p( x \middle| \theta_{i} )} = {\frac{1}{2\pi^{p/2}{\sum }^{1/2}}{\mathbb{e}}^{{- \frac{1}{2}}{({x - \mu_{i}})}^{T}{\sum\limits^{- 1}{({x - \mu_{i}})}}}}},{i = 1},2} & (4)\end{matrix}$

P300 is identified using Fisher's linear discriminant in the exampleembodiment. The Fisher's linear discriminant is chosen because of itslower computational cost and its superior performance as compared toother P300 identification techniques as reported in [D. J. Krusienski,E. W. Sellers, F. Cabestaing, S. Bayoudh, D. J. McFarland, T. M.Vaughan, J. R. Wolpaw, “A comparison of classification techniques forthe P300 Speller,” Journal of neural engineering, vol. 3, no. 34, pp.299-305, 2006.]. However, other P300 identification techniques such asthe Pearson's correlation method, stepwise linear discriminant analysis,and Support Vector Machine (SVM) can also be used.

The Fisher's linear discriminant in the example embodiment attempts tofind a linear combination w that projects a high-dimensional feature xinto a one-dimensional feature g(x) according to Equation (5). InEquation (5), w and w₀ refer to the weight vector and bias. For thetwo-class case, the linear discriminant g₁(x)=g₂(x) defines a boundarysurface, which is a hyperplane whose orientation is determined by the win Equation (5).g(x)=w ^(T) x+w ₀  (5)

The Fisher's linear discriminant in the example embodiment seeks todetermine a linear combination of the feature vector x that maximizesthe ratio of its between-classes variance to its within-classes varianceaccording to Equation (6). In Equation (6), S_(b) and S_(w) correspondto the between-classes scatter matrix and within-classes scatter matrix,respectively whereas J(w) is the generalized Rayleight quotient. For thetwo-class case, the two scatter matrices in Equation (6) can beestimated from the training EEG according to Equation (7). In Equation(7), μ₁ and μ₂ refer to the mean of the EEG feature vectors with andwithout P300, respectively.

$\begin{matrix}{{\underset{w}{\arg\;\max}\;{J(w)}} = \frac{w^{T}S_{b}w}{w^{T}S_{w}w}} & (6) \\{{S_{w} = {\sum\limits_{i = 1}^{2}{\sum\limits_{x \in D_{i}}^{\;}{( {x - \mu_{i}} )( {x - \mu_{i}} )^{T}}}}},{S_{b} = {( {\mu_{1} - \mu_{2}} )( {\mu_{1} - \mu_{2}} )^{T}}},{i = 1},2} & (7)\end{matrix}$

The orientation of the boundary surface w (weight vector) that maximizesthe quantity J(w) can be determined according to Equation (8). As thefeature vector x is distributed normally, the weight vector w inEquation (8) can be similarly derived by the discriminant function thatmaximizes the posterior probability g_(i)(x) according to Equation (9).In Equation (9), p(θ_(i)|x) refers to the posterior probability thatgiven the hypothesis θ_(i), the feature vector x is obtained wherebyθ_(i) represents the hypothesis that the EEG contains P300 when i=1 andrepresents the hypothesis that the EEG does not contain P300 when i=2.In short, p(θ_(i)|x) is the P300 and the non-P300 posterior probabilityof the feature vector x for i=1 and 2 respectively. Furthermore,p(θ_(i)) refers to a priori probability and in one example, p(θ_(i)) isequal to ⅙ and ⅚ for i=1 and 2 respectively. In addition, p(x|θ_(i)),i=1, 2 follows a multivariate Gaussian distribution according toEquation (4) and the parameters, μ and Σ, can be estimated from thefeature vector x converted from the training EEG.w=S _(w) ⁻¹(μ₁−μ₂)  (8)g _(i)(x)=ln p(θ_(i) |x)=ln p(x|θ _(i))+ln(p(θ_(i))), i=1,2  (9)

To build the SIM model, the pooled labelled subject EEG is convertedinto a set of feature vectors and the corresponding labels according toEquation (10). In Equation (10), x_(si) and l_(si) refer to the featurevector converted from EEG of the i-th subject and its correspondinglabeling, respectively.X={[x _(sl) ^(T) ,l _(sl) ], . . . ,[x _(si) ^(T) ,l _(si) ], . . . ,[x_(sn) ^(T) ,l _(sn)]}  (10)

With the pooled EEG X in Equation (10), the Gaussian distributionp(x|θ_(i)) (μ_(i) and covariance Σ_(i)) in Equation (9) can be estimatedand a SIM can then be built based on Fisher's linear discriminant.

Unlabelled subject EEG, E′, from a new subject is divided into a numberof roughly equal segments i.e. segmented and is preprocessed in step208. In one example, the preprocessing in step 208 is the same as thatin step 202.

In step 210, feature extraction is performed whereby the segmented andpreprocessed EEG is converted into a set of feature vectors. In oneexample, the set of feature vectors can be expressed in the formX′={x_(s) _(i) , . . . x_(s) _(l) , . . . x_(s) _(n) } where x_(s) _(l)is the feature vector corresponding to the i-th EEG segment of the newsubject.

In step 212, an initial SSM is built based on the feature vector x_(s)_(l) corresponding to the first segment of the EEG of the new subjectand its corresponding label l_(s) _(l) whereby the corresponding labelis predicted by classifying the feature vector x_(s) _(l) using the SIMbuilt in step 206. Therefore, the EEG of a new subject can be classifiedand the initial SSM can be built using the SIM without special training.

In step 214, unsupervised modeling and classification is performed tooutput an adapted SSM and to give the final classification results. Inthe example embodiments, except for the first EEG segment of the newsubject whose labels are solely predicted by the SIM (i.e. during thebuilding of the initial SSM), the ensuing EEG segments of the newsubject are all classified by both the SIM and the SSM in step 214. Oneexample of step 214 is illustrated in FIG. 3.

FIG. 3 shows a flowchart illustrating an unsupervised modeling andclassification technique 300 according to an embodiment of the presentinvention. In step 302, the feature vector of the ensuing EEG segment ofthe new subject, x_(s) _(i) is classified using the SIM whereas in step304, the feature vector of the ensuing EEG segment of the new subject,x_(s) _(i) , is classified using the SSM.

In step 306, SIM confidence evaluation is performed whereas in step 308SSM confidence evaluation is performed. Further details of steps 306 and308 are as follows.

To facilitate the ensuing SSM adaptation, the P300 and non-P300posterior probabilities of the feature vector x in Equation (9) i.e. g₁and g₂ respectively, are transformed according to Equation (11). InEquation (11), G=[g₁, g₂] corresponds to the posterior probability ofthere being a P300 and no P300 as evaluated by Equation (9). Thus thetransformation in Equation (11) maps the posterior probability to atransformed posterior probability with a value between 0 and 1.

$\begin{matrix}{\phi_{i}\frac{{\mathbb{e}}^{g_{i} - {\min{(G)}}}}{\sum\limits_{i = 1}^{2}{\mathbb{e}}^{g_{i} - {\min{(G)}}}}} & (11)\end{matrix}$

In steps 306 and 308, scores are defined to indicate the models'confidence in the P300 identification. In one example, the models'confidence scores are defined based on the transformed P300 posteriorprobability (as evaluated in Equation (11)) according to Equation (12).In Equation (12), φ_(1,rmax) and φ_(1,cmax) are the maximum P300posterior probabilities among the flashing rows and columns respectivelywhereas φ′_(1,rmax) and φ′_(1,rmax) are the second maximum P300posterior probabilities among the flashing rows and columnsrespectively. As shown in Equation (12), the confidence score is highwhen the maximum P300 posterior probability and the saliency of themaximum. P300 posterior probability (given by the difference between themaximum P300 posterior probability and second maximum P300 posteriorprobability) are both high.conf=(φ_(1,rmax)−φ′_(1,rmax))+(φ_(1,cmax)−φ′_(1,cmax))  (12)

In step 310, the model with the higher confidence score is selected toclassify x_(s) _(i) for the i-th segment and the label of x_(s) _(i)i.e. (l_(s) _(i) ) is predicted by either the SIM or the SSM dependingon which model has a higher confidence score. In one example, the labelsof the ensuing EEG segments of the new subject are determined based onthe confidence scores of the SIM and SSM according to Equation (13). InEquation (13), conf_(SIM) and conf_(SSM) refer to the confidence scoresof the SIM and the SSM (as evaluated by Equation (13)), respectively andl_(sim) and l_(ssm) refer to the labels predicted by the SIM and SSMrespectively.

$\begin{matrix}{l_{s_{i}}\{ {\begin{matrix}l_{sim} \\l_{ssm}\end{matrix}\begin{matrix}{{{{if}\mspace{14mu}{conf}_{SIM}} \geq {conf}_{SSM}}\mspace{14mu}} \\{otherwise}\end{matrix}} } & (13)\end{matrix}$

In step 312, it is determined if the SSM has achieved a consistentconfidence score. If so, the final classification results are theclassification results obtained in step 310. If it is determined in step312 that the SSM has not achieved a consistent confidence score, SSMadaptation is performed in step 314. In step 314, the SSM is updated togive an adapted SSM using all the EEG segments, x_(s) ₁ , . . . , x_(s)_(i) classified so far and their corresponding labels determined in step310.

In one example, the adaptation of the SSM in the example embodiment isterminated when the SSM has achieved a consistent confidence scoreaccording to Equation (14). If the conditions in Equation (14) aresatisfied, it is determined that the SSM has achieved consistency. InEquation (14), conf_(i,SSM) and conf_(i-k,SSM) refer to the confidencescores of the just classified and the last k classified EEG segments ofthe new subject, respectively. Generally, the parameter k can be anumber lying between 3 and 8. The number is set at 5 in this exampleimplementation. Furthermore, parameters T₁ and T₂ refer to predeterminedthresholds for the confidence score, conf_(i,SSM) and the confidenceconsistency S([conf_(i-k,SSM), . . . , conf_(i,SSM)]) respectively. Thefunction S( ) evaluates the standard deviation of the input vector[conf_(i-k,SSM), . . . , conf_(i,SSM)] whereby this input vectorrepresents the confidence scores of multiple consecutive EEG segments.

$\begin{matrix}\{ \begin{matrix}{{conf}_{i,{SSM}} > T_{1}} \\{S( {\lbrack {{conf}_{{i - k},{SSM}},\ldots\mspace{14mu},{conf}_{i,{SSM}}} \rbrack < T_{2}} )}\end{matrix}  & (14)\end{matrix}$

Steps 302 to 314 are repeated until it is determined in step 312 thatthe SSM has achieved a certain level of consistency and confidence. Theoutput of FIG. 3 is hence an adapted SSM capable of classifying EEG ofthe new subject and the final classification results of the subject EEGgiven by this adapted SSM during the model adaptation.

In the example embodiment, multiple rounds of flashing are implementedfor the subject to input one character and the focused cell isidentified by the row flash and the column flash that resulted in themaximum averaged P300 posterior probability over multiple rounds offlashing (i.e. row_(P300) and col_(P300) respectively) according toEquation (15). In Equation (15), φ_(1,irow,j) and φ_(1,icol,j) arerespectively the posterior probabilities that P300 is evoked during theirow-th and the icol-th flash within the j-th round of flashing. R isthe number of rounds implemented and is equal to 10 in one example.

$\begin{matrix}\begin{matrix}{{row}_{P\; 300} = {\underset{irow}{\arg\;\max}{\sum\limits_{{irow} = 1}^{6}{\sum\limits_{j = 1}^{R}\phi_{1,{irow},j}}}}} \\{{col}_{P300} = {\underset{icol}{\arg\;\max}{\sum\limits_{{icol} = 1}^{6}{\sum\limits_{j = 1}^{R}\phi_{1,{icol},j}}}}}\end{matrix} & (15)\end{matrix}$

The advantages of the example embodiments include the following.

The technique in the example embodiment adapts a SIM to a SSM in anunsupervised manner and is superior to semi-supervised learningtechniques [Y. Li, H. Li, C. Guan and Z. Chin, A self-trainingsemi-supervised support vector and its applications in brain computerinterface, IEEE International Conference on Acoustics, Speech and SignalPro., pp. 385-388, 2007.], which are used when only a small amount oflabelled data are available. In contrast to the semi-supervised learningtechniques, the technique in the example embodiments requires no EEGlabels of the new subject. Instead, it uses the SIM as a seed model tomake an initial label prediction. Furthermore, the example embodimentincludes a new P300 modeling and identification technique that adaptsthe subject-independent P300 model (i.e. SIM) to a SSM in anunsupervised manner.

To further illustrate the advantages of embodiments of the presentinvention, experimental results from the implementation of the exampleembodiment of the present invention are presented below.

In the experiment, the technique in the described example embodiment istested over a P300-based word speller. Further details of the P300-basedword speller can be found in [T. Manoj, C. Guan, and W. Jiankang. Robustclassification of EEG signal for brain-computer interface, IEEETransactions on Neural Systems and Rehabilitation Engineering, vol. 14,no. 1, pp. 24-29, 2006.], the contents of which are incorporated hereinby cross-reference. In this speller system in the experiment; subjectsare equipped with an electro-cap that has 64 electrodes. The subject EEGis first amplified by a Neuroscan amplifier called SynAmps2 and is thenpiped to a server by the Neuroscan software. 24 out of the 64 channelsof the SynAmps2 are selected and the EEG sampling rate is set at 250 Hz[C. Guan, X. Zhu, S. Ranganatha, M. Thulasidas, and J. Wu, Robustclassification of event-related potential for brain-computer interface,Int. Conf. Adv. Medical Signal Info. Processing, pp. 321-326, 2004.].

In the experiment, during the EEG collection stage, the subjects sit infront of a 6×6 matrix of cells with each cell displaying a character asshown in FIG. 1 where the six rows and columns flash successively andrandomly. Subjects are required to focus on one specific cell visuallyduring a flashing round. When a particular row or column flashes, acorresponding stimulus code is generated in a time-locked fashion, whichdivides the collected EEG into epochs of 500 ms starting from the timethe stimulus appears. Therefore, the focused cell corresponding to onerow flash and one column flash within each round can be identifiedthrough the identification of the P300 within the epoched EEG.

Furthermore, in the experiment, the EEG of ten healthy subjects iscollected. For each subject, two EEG sessions are collectedsequentially, which correspond to the input of the same set of 41characters “THE QUICK BROWN FOX JUMPS OVER LAZY DOG 246138 579” in twodifferent orders. In addition, ten rounds of flashes are implemented foreach character. Within each round, the EEG between 150 ms and 500 msfollowing each flash are used for the P300 identification. These twosessions of EEG are used to evaluate the P300 identification techniquein the experiment.

In one experiment, the P300 variability is studied. Tough P300 iscommonly defined by a positive peak after 300 ms of elicited stimuliwhereas the real P300 usually varies greatly from subject to subject interms of its peak amplitude and its peak latency. Consequently, a P300model learned from one subject usually cannot apply well to anothersubject.

In the experiment, the P300 variability is studied through theexamination of the cross-subject EEG classification. First, ten subjectmodels are built by learning from the first session of EEG (or thesecond session depending on the two-fold cross validation) of the tenhealthy subjects. Subsequently, the ten subject models are then used toclassify the second session of EEG (or the first session depending onthe two-fold cross validation) of the ten healthy subjects.

Table 1 shows the cross-subject P300 identification accuracies. Inparticular, the rows in Table 1 represent the ten subject models and thecolumns represent the second (or the first) session of EEG of the tenhealthy subjects to be classified. Therefore, the diagonal items inTable 1 show the subject-specific accuracies, which are evaluated byusing the models learned from the subject's own EEG to classify thesecond (or first) session of EEG whereas the non-diagonal items give thecross-subject accuracies, which are evaluated by using the models thatare learned from EEG of other subjects to classify the second (or first)session of EEG.

TABLE 1 Subj. 1 Subj. 2 Subj. 3 Subj. 4 Subj. 5 Subj. 6 Subj. 7 Subj. 8Subj. 9 Subj. 10 Model 1 0.9878 0.8049 0.9268 0.8659 0.4146 0.82930.7195 0.6951 0.8537 0.4756 Model 2 0.9024 0.9756 0.8293 0.5854 0.29270.4756 0.3659 0.4024 0.8415 0.4268 Model 3 0.9146 0.7317 0.9878 0.85370.2561 0.8415 0.6341 0.7073 0.6829 0.6341 Model 4 0.7805 0.4634 0.76831.0000 0.4024 0.8902 0.5976 0.4878 0.8780 0.3659 Model 5 0.4878 0.28050.5000 0.6341 0.9024 0.6829 0.5244 0.6707 0.4024 0.1707 Model 6 0.58540.1463 0.6585 0.8659 0.4024 1.0000 0.9512 0.9634 0.6829 0.1951 Model 70.5488 0.2805 0.6585 0.8780 0.3659 0.8537 1.0000 0.8780 0.6585 0.2317Model 8 0.2439 0.2073 0.4390 0.5000 0.1098 0.8902 0.6585 1.0000 0.20730.3049 Model 9 0.8902 0.8902 0.9024 0.9878 0.5366 0.9268 0.8293 0.81711.0000 0.5244 Model 10 0.8415 0.6220 0.7195 0.5244 0.1463 0.5366 0.62200.7805 0.5610 0.9512

In addition, FIG. 4 shows a plot 400 illustrating the performances ofdifferent P300 identification techniques. In FIG. 4, the graph 402labelled by circles shows the accuracy of the subject-specific P300models corresponding to the diagonal items in Table 1 and the graph 406labelled by diamonds shows the cross-subject accuracy that is derived byaveraging the non-diagonal items in Table I in each column.

From Table 1 and FIG. 4, it can be seen that the cross-subject accuracyas shown by graph 406 (non-diagonal items in Table 1) is significantlylower than the subject-specific accuracy as shown by graph 402 (diagonalitems in Table 1), indicating the EEG variability among differentsubjects.

In FIG. 4, the subject-independent P300 identification technique i.e.SIM of an example embodiment is also tested. For each of the ten healthysubjects, a SIM is built by learning from the EEG of the other ninesubjects according to Equation (16). In Equation (16), TR_(j) refers tothe first session of EEG (or the second session of EEG depending ontwo-fold cross-validation) of the j-th subject and P_(i) refers to theEEG of the other nine subjects used to train a SIM of a subject understudy. The trained SIM is then used to classify the second session ofEEG (or the first session of EEG depending on two-fold cross-validation)of the i-th subject.P _(i) ={TR _(j), for j=1 . . . 10, where j≠i}  (16)

In FIG. 4, the graph 404 labelled by squares shows the accuracy of theSIM for each subject. As shown in graph 404, the accuracy of the SIMs isgenerally much higher than that of the cross-subject models. While theaccuracy of the SIMs is still lower than that of the SSMs, the resultsindicate that the combination of EEG of a pool of subjects in thisexample embodiment advantageously augments the performance of the EEGclassification greatly, compared to the use of cross-subject models.

In the experiment, to remove the possible effects of the EEG collectionorder, 41 characters in one session are randomly sorted and subsequentlydivided into 40 segments including 2 characters in the first segment and1 in each of the remaining 39 segments. In addition, to get morecomprehensive classification results, ten rounds of the random charactersorting and segmenting described above are implemented for each of thetwo sessions, and the graphs shown in FIG. 4 relate to results after thetenths round. Therefore, 20 sets of SSMs are built for each subjectwhereby each set is composed of 40 SSMs.

For comparison in the experiment, 20 sets of supervised SSMs (40 in eachset) are built for each subject based on the 20 sets of randomly sortedand segmented EEG as described above. In particular, the i-th supervisedSSM in each set is built by learning from the first i segments of thesubject EEG together with the corresponding labels. The adapted SSMs inthe example embodiments and the supervised SSMs for comparison are thenused to classify other sessions of the subject EEG.

It should be noted that the use of 3 characters in the first segment asdescribed above is preferable as it can improve the robustness of thetechnique in the example embodiments greatly. This is because if only asingle character is included in the first segment and most of the EEGused to input that single character is incidentally misclassified by theSIM, the initially adapted SSM may not be able to capture thesubject-specific P300 characteristics and the adaptation may not resultin a good SSM eventually.

FIGS. 5A-5J show graphs illustrating the P300 identification accuracy ofsupervised SSMs, the SIMs according to an example embodiment, and theadapted SSMs according to another embodiment of the present invention.In FIGS. 5A-J, the results for the ten healthy subjects averaged overthe 20 sets of subject models are shown. In particular, the solid graphs502A, 502B, 502C, 502D, 502E, 502F, 502G, 502H, 502I and 502J show theaccuracy of the supervised SSMs whereas the dotted graphs 504A, 504B,504C, 504D, 504E, 504F, 504G, 504H, 504I and 504J show the accuracy ofthe adapted SSMs in the example embodiment. As can be see from FIG. 5,while the accuracy of the adapted SSMs in the example embodiment isinitially lower than that of the supervised SSMs for most of thesubjects, the accuracy increases steadily and quickly converges to theaccuracy of the supervised SSMs when around 25 characters areincorporated. In addition, even for the fifth subject whose EEG is quitedifferent from the others as depicted in FIG. 5E, the accuracy of theadapted SSM in the example embodiment can also converge to the accuracyof the supervised SSM.

The accuracy in FIG. 5 is evaluated when ten rounds of test EEG data areused for each character. It is noted that the model accuracy is improvedwhen more and more training data is used for supervised SSMs or adaptedSSMs. For the SIMs, the accuracy does not change because the SIM isfixed (trained by the pooled EEG). The axis of the figures in FIG. 5labelled “Number of characters” refers to the amount of training EEGdata used for the training of the supervised and adapted SSMs, whereeach character corresponds to ten rounds of EEG that are used to spellthe character. As described above for the test data, the training EEGdata is likewise divided into 40 segments including 2 characterscorresponding to 2*10 rounds of EEG in the first and 1 character in eachthereafter in the example embodiment.

FIGS. 6A-J show graphs illustrating the standard deviation of the P300identification accuracy of the supervised SSMs, and the adapted SSMsaccording to an embodiment of the present invention. For the ten healthysubjects, the graphs in FIG. 6 show the standard deviations of the modelaccuracy evaluated across the 20 sets of SSMs as described above. Thesolid graphs 602A, 602B, 602C, 602D, 602E, 602F, 602G, 602H, 602I and602J show the standard deviation of the accuracy of the supervised SSMswhereas the dotted graphs 604A, 604B, 604C, 604D, 604E, 604F, 604G,604H, 604I and 604J show the standard deviation of the accuracy of theadapted SSMs. As shown in FIG. 6, the standard deviation of the accuracyof the adapted SSMs is quite close to that of the supervised SSMs formost of the ten subjects (except for the fifth subject as depicted inFIG. 5E). At the same time, the accuracy variance of the adapted SSMs isquite small (except for the fifth subject as depicted in FIG. 5E),indicating the stability of the technique in this example embodiment.The fifth subject's much larger accuracy deviation may be because hisP300 differs substantially from that of the other nine subjects.

Both the supervised and adapted SSMs are trained by increasing thenumber of the training characters step by step in this comparison.Particularly, the 41 training characters are divided into 40 segments (2characters in the first segment and 1 in each of the remaining 39segments) and train the SSMs (for both supervised and adapted) with thetraining character segments being increased from 1 to 40.

The trained SSMs are then used to classify the test data (another EEGsession composed of 41 characters) as illustrated in FIGS. 5 and 6. Ascan be seen from FIGS. 5 and 6, the SSM accuracy (evaluated over thetest data) increases steadily when more EEG (from the training EEG) isused for the model training.

FIG. 7 shows a plot 700 illustrating the classification accuracy ofsupervised SSMs, adapted SSMs according to an example embodiment, andSIMs according to another embodiment of the present invention overdifferent rounds of intensification. In FIG. 7, graph 702 shows theclassification accuracy of the supervised SSMs over different rounds ofintensification, graphs 704 and 706 respectively show the classificationaccuracy of the adapted SSMs in the example embodiments when 20% of thedata (8 characters) and 30% of the data (12 characters) are adapted andgraph 708 shows the classification accuracy of the SIMs in the exampleembodiment over different rounds of intensification. As shown in FIG. 7,with a small amount of subject EEG incorporated during the SSM upgradingprocess, compared to supervised SSMs, the adapted SSM is capable ofachieving virtually the same performance as the supervised SSM trainedby labelled subject EEG.

It is noted that the accuracy of both SIMs and SSMs will increase when alarger number of rounds of EEG are used for the classification. Due tothe noise, the single-round EEG classification accuracy is very low asillustrated in FIG. 7. To solve this problem, most existing BCIs collectEEG in multiple rounds and then suppress the noise effect throughaveraging the EEG collected in multiple rounds.

From the experimental results it can be seen that the SIM in an exampleembodiment outperforms the cross-subject model significantly.Furthermore, the SSM adapted with unlabelled subject EEG in anotherexample embodiment not only outperforms the SIM but is also capable ofachieving virtually the same performance as the supervised SSM trainedby labelled subject EEG. This is achieved by using only a small amountof subject EEG incorporated during the SSM upgrading process, comparedto the supervised SSM process. Compared with using the supervised SSMs,using the SIM and adapted SSMs in the example embodiments can remove thetedious and complicated training procedure. Therefore, P300-based BCIscan be made to be user-friendlier and more easily implemented.

Furthermore, the techniques in the example embodiments are not limitedto identifying P300 using a P300-based word speller. The technique oflearning a SIM from a pool of subjects and the adaptation of a SIM to aSSM in example embodiments can also be applied to other EEG-based BCIssuch as those using motor imagery.

Hence, in one example embodiment, an adaptive EEG classificationtechnique has been developed. In this technique in the exampleembodiment, a SIM is first built, which itself, as an embodiment of theinvention, augments the classification of EEG of a new subject bylearning from a pool of existing subjects. Next, the SIM is adapted to aSSM for a new subject through an unsupervised learning process. Withapplication to a P300 word speller, experiments over ten healthysubjects show that the adapted SSM is capable of achieving virtually thesame performance as the supervised SSM trained by labelled subject EEG.Hence, the use of the adapted SSM in this example embodiment can removethe complicated and tedious training process without compromising on itsperformance.

FIG. 8 illustrates a schematic block diagram of a system 800 forclassifying brain signals in the BCI according to an embodiment of thepresent invention. The system 800 includes an input unit 802 forreceiving brain signals and a model building unit 804 for building asubject-independent model using labelled brain signals from a pool ofsubjects.

The method and system of the example embodiment can be implemented on acomputer system 900, schematically shown in FIG. 9. It may beimplemented as software, such as a computer program being executedwithin the computer system 900, and instructing the computer system 900to conduct the method of the example embodiment.

The computer system 900 comprises a computer module 902, input modulessuch as a keyboard 904 and mouse 906 and a plurality of output devicessuch as a display 908, and printer 910.

The computer module 902 is connected to a computer network 912 via asuitable transceiver device 914, to enable access to e.g. the Internetor other network systems such as Local Area Network (LAN) or Wide AreaNetwork (WAN).

The computer module 902 in the example includes a processor 918, aRandom Access Memory (RAM) 920 and a Read Only Memory (ROM) 922. Thecomputer module 902 also includes a number of Input/Output (I/O)interfaces, for example I/O interface 924 to the display 908, and I/Ointerface 926 to the keyboard 904.

The components of the computer module 902 typically communicate via aninterconnected bus 928 and in a manner known to the person skilled inthe relevant art.

The application program is typically supplied to the user of thecomputer system 900 encoded on a data storage medium such as a CD-ROM orflash memory carrier and read utilising a corresponding data storagemedium drive of a data storage device 930. The application program isread and controlled in its execution by the processor 91B. Intermediatestorage of program data maybe accomplished using RAM 920.

A method of classifying brain signals in a BCI according to anembodiment of the present invention comprises building asubject-independent model using labelled brain signals from a pool ofsubjects.

It will be appreciated by a person skilled in the art that numerousvariations and/or modifications may be made to the present invention asshown in the specific embodiments without departing from the spirit orscope of the invention as broadly described. The present embodimentsare, therefore, to be considered in all respects to be illustrative andnot restrictive. For example, while the use of EEG has been described inthe example embodiments of the present invention, other types of brainsignals such as MEG signals or a mixture of both MEG and EEG signals canalso be used.

What is claimed is:
 1. A method for classifying brain signals in abrain-computer interface (BCI), the method comprising the steps of:building a subject-independent model for a new subject using labelledbrain signals from a pool of previous subjects, and building an initialsubject-specific model for the new subject based on thesubject-independent model and a segment of the unlabelled brain signalsfrom the new subject.
 2. The method as claimed in claim 1, wherein thestep of building an initial subject-specific model is based on thesubject-independent model and a set of feature vectors extracted fromsaid segment of the unlabelled brain signals from the new subject, saidsegment being a first segment of the unlabelled brain signals, and themethod further comprises applying both the subject-independent model andthe initial subject-specific model for classifying subsequent segmentsof the unlabelled brain signals.
 3. The method as claimed in claim 2,further comprising the step of adapting the initial subject-specificmodel using one or more of a group consisting of the subsequent segmentsof unlabelled brain signals from the new subject, thesubject-independent model and the initial subject-specific model.
 4. Themethod as claimed in claim 3, wherein the adapting is performed untilthe subject-specific model achieves a consistent confidence score andsubsequently the adapted subject specific model is used to give theclassification of the brain signals.
 5. The method as claimed in claim1, wherein the step of building the subject-independent model usinglabelled brain signals from a pool of previous subjects furthercomprises the steps of: acquiring the labelled brain signals from thepool of previous subjects; preprocessing the acquired labelled brainsignals; constructing a set of feature vectors with their correspondinglabels from the preprocessed brain signals; and building thesubject-independent model by finding a weight vector for a linearcombination of each feature vector to maximize the posterior probabilitythat a P300 is evoked or not evoked given a feature vector.
 6. Themethod as claimed in claim 2, wherein the step of building the initialsubject-specific model comprises the steps of: acquiring the unlabelledbrain signals from the new subject; segmenting the acquired unlabelledbrain signals; preprocessing the acquired unlabelled brain signals;extracting a set of feature vectors from the preprocessed unlabelledbrain signals; and classifying the first segment of the unlabelled brainsignals using the subject-independent model to build the initialsubject-specific model.
 7. The method as claimed in claim 5, wherein thestep of acquiring the labelled brain signals from the pool of previoussubjects further comprises the steps of: providing a pre-defined set ofstimuli in rows and columns; repeatedly activating the stimuli inrounds, wherein in each round, each row or column of stimuli isactivated once; acquiring brain signals from the pool of previoussubjects with each previous subject focused on a known stimulus; andlabelling the acquired brain signals from the pool of previous subjectsusing the label of the known stimulus to give the labelled brainsignals.
 8. The method as claimed in claim 6, wherein the step ofacquiring the unlabelled brain signals from the new subject furthercomprises the steps of: providing a pre-defined set of stimuli in rowsand columns; repeatedly activating the stimuli in rounds, wherein ineach round, each row or column of stimuli is activated once; andacquiring the unlabelled brain signals from the new subject with thesubject focused on an unknown stimulus.
 9. The method as claimed inclaim 5, wherein the step of preprocessing the acquired labelled brainsignals further comprises the steps of: implementing a low-passfiltering of the acquired labelled brain signals using an optimal cutofffrequency; down-sampling the filtered brain signals by averaging everyfive consecutive samples to a single sample; and removing ocularartifacts from the downsampled brain signals.
 10. The method as claimedin claim 6, wherein the step of preprocessing the acquired unlabelledbrain signals further comprises the steps of: implementing a low-passfiltering of the brain signals using an optimal cutoff frequency;down-sampling the filtered brain signals by averaging every fiveconsecutive samples to a single sample; and removing ocular artifactsfrom the downsampled brain signals.
 11. The method as claimed in claim 6wherein the step of segmenting the acquired unlabelled brain signalsfurther comprises the step of including brain signals collected for morethan one stimulus in the first segment and including brain signalscollected for one stimulus in each of the subsequent segments.
 12. Themethod as claimed in claim 3, wherein the step of adapting the initialsubject-specific model using one or more of a group consisting of thesubsequent segments of unlabelled brain signals from the new subject,the subject-independent model and the initial subject-specific modelfurther comprises the steps of iteratively: a) classifying the featurevectors corresponding to the subsequent segment of the unlabelled brainsignals using the subject-independent model; b) classifying the featurevectors corresponding to the subsequent segment of the unlabelled brainsignals using the initial subject-specific model; c) evaluating aconfidence score for the subject-independent model; d) evaluating aconfidence score for the initial subject-specific model; e) classifyingthe feature vector corresponding to the subsequent segment of theunlabelled brain signals using the model with a higher confidence score;f) determining if the initial subject-specific model has achieved aconsistent confidence score; g) adapting the initial subject-specificmodel using classification results from the model with a higherconfidence score if the subject-specific model has not achieved aconsistent confidence score; and repeating steps a) to g) with theadapted initial subject-specific model as the initial subject-specificmodel.
 13. The method as claimed in claim 12, wherein the step ofevaluating the confidence score for the subject-independent modelfurther comprises the steps of: evaluating a posterior probability thata P300 is evoked given the feature vector for each row of stimuli;evaluating a posterior probability that a P300 is evoked given thefeature vector for each column of stimuli; determining the differencebetween the highest posterior probability among the rows of stimuli andthe next highest posterior probability among the rows of stimuli to givea saliency of the highest posterior probability and multiplying saidsaliency to said difference; determining the difference between thehighest posterior probability among the columns of stimuli and the nexthighest posterior probability among the columns of stimuli to give asaliency of the highest posterior probability and multiplying saidsaliency to said difference; and combining the product of the saliencyand the difference for the rows of stimuli and the columns of stimuli toevaluate a confidence score for the subject-independent model.
 14. Themethod as claimed in claim 12, wherein the step of evaluating theconfidence score for the initial subject-specific model furthercomprises the steps of: evaluating a posterior probability that a P300is evoked given the feature vector for each row of stimuli; evaluating aposterior probability that a P300 is evoked given the feature vector foreach column of stimuli; determining the difference between the highestposterior probability among the rows of stimuli and the next highestposterior probability among the rows of stimuli to give a saliency ofthe highest posterior probability and multiplying said saliency to saiddifference; determining the difference between the highest posteriorprobability among the columns of stimuli and the next highest posteriorprobability among the columns of stimuli to give a saliency of thehighest posterior probability and multiplying said saliency to saiddifference; and combining the product of the saliency and the differencefor the rows of stimuli and the columns of stimuli to evaluate aconfidence score for the initial subject-specific model.
 15. The methodas claimed in claim 12, wherein the step of determining if the initialsubject-specific model has achieved a consistent confidence scorefurther comprises the steps of: determining if the confidence score ofthe initial subject-specific model for a current segment of brainsignals from the new subject is greater than a first threshold;determining if the standard deviation of the confidence scores of theinitial subject-specific model for the last k segments of brain signalsfrom the new subject is less than a second threshold; and determiningthat the initial subject-specific model has achieved a consistentconfidence score if the confidence score of the initial subject-specificmodel for a current segment of brain signals is greater than said firstthreshold and the standard deviation of the confidence scores of theinitial subject-specific model for the last k segments of brain signalsis less than said second threshold.
 16. The method as claimed in claim8, further comprising the step of identifying the unknown stimulus byidentifying the row and the column in which the unknown stimulus lieswherein the stimulus in said row and said column results in a maximumaveraged posterior probability that P300 is evoked given a featurevector.
 17. A system for classifying brain signals in a brain-computerinterface (BCI), the system comprising: a first model building unit forbuilding a subject-independent model for a new subject using labelledbrain signals from a pool of previous subjects, and a second modelbuilding unit for building an initial subject-specific model for the newsubject based on the subject-independent model and a segment of theunlabelled brain signals from the new subject.
 18. The system as claimedin claim 17, wherein the second model building unit for building aninitial subject-specific model is based on the subject-independent modeland a set of feature vectors extracted from said segment of theunlabelled brain signals from the new subject, said segment being afirst segment of the unlabelled brain signals, and the system isconfigured to apply both the subject-independent model and the initialsubject-specific model for classifying subsequent segments of theunlabelled brain signals.
 19. The system as claimed in claim 18, furthercomprising a model adapting unit for adapting the initialsubject-specific model using one or more of a group consisting of thesubsequent segments of unlabelled brain signals from the new subject,the subject-independent model and the initial subject-specific model.20. The system as claimed in claim 19, wherein the adapting is performeduntil the subject-specific model achieves a consistent confidence scoreand subsequently the adapted subject specific model is used to give theclassification of the brain signals.
 21. The system as claimed in claim17, further comprising: a stimulation unit comprising a set of stimuliin rows and columns; wherein the stimulation unit repeatedly activatesthe stimuli in rounds, such that in each round, each row or column ofstimuli is activated once; an acquisition unit for acquiring brainsignals; and a preprocessing unit for preprocessing the acquired brainsignals.
 22. A non-transitory data storage medium having stored thereoncomputer code means for instructing a computer system to execute amethod for classifying brain signals in a brain-computer interface(BCI), the method comprising the steps of: building asubject-independent model using labelled brain signals for a new subjectfrom a pool of previous subjects, and building an initialsubject-specific model for the new subject based on thesubject-independent model and a segment of the unlabelled brain signalsfrom the new subject.
 23. The non-transitory data storage medium asclaimed in claim 22, wherein the step of building an initialsubject-specific model is based on the subject-independent model and aset of feature vectors extracted from said segment of the unlabelledbrain signals from the new subject, said segment being a first segmentof the unlabelled brain signals, the method further comprises applyingboth the subject-independent model and the initial subject-specificmodel for classifying the subsequent segments of the unlabelled brainsignals.
 24. The non-transitory data storage medium as claimed in claim23, the method further comprising the step of adapting the initialsubject-specific model using one or more of a group consisting of thesubsequent segments of unlabelled brain signals from the new subject,the subject-independent model and the initial subject-specific model.25. The non-transitory data storage medium as claimed in claim 24,wherein the adapting is performed until the subject-specific modelachieves a consistent confidence score and subsequently the adaptedsubject specific model is used to give the classification of the brainsignals.